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ABSTRACT 

Objectives: Hospital discharge data (HDD) in many 
health systems do not capture the date of injury (D0I); 
the absence of this date hinders researchers' ability to 
distinguish repeat from incident injury admissions. 
Various approaches using somewhat arbitrary criteria 
have been explored to increase the accuracy of incident 
injury identification. However, these approaches have 
not been validated against a data source which 
contains DOI. The aim of this study was to determine 
the accuracy of evidence-based criteria for 
identifying fall-related incident hip fractures in the 
absence of DOI using HDD containing DOI as 
the reference standard. 
Design: Retrospective database study. 
Setting: New Zealand. 

Participants: 8761 patients aged 65+ years admitted 
for fall-related hip fracture between 1 July 2005 and 30 
June 2008, inclusive. 

Outcome measures: We defined person-identifying 
HDD containing DOI as the reference standard and 
calculated measures of the accuracy of evidence-based 
criteria for identifying fall-related incident hip fractures 
from HDD not containing DOI. The criteria were 
principal diagnosis of hip fracture, mechanism of injury 
indicating a fall, admission type emergency, admission 
source other than a transfer and presence of hip 
operation code(s). For a subsequent fall-related hip 
fracture, additional criteria were time between 
successive hip fractures >120 days, and all external 
cause-of-injury codes being different to those for the 
previous hip fracture. 

Results: The sensitivity and specificity of the criteria 
for identifying fall-related incident hip fractures from 
data not containing DOI were 96.7% and 99.3%, 
respectively, compared with the reference standard. 
The application of these criteria resulted in a slight 
underestimation of the percentage of patients with 
multiple hip fractures. 

Conclusions: Although it is preferable to have DOI; 
this study demonstrates that evidence-based criteria 
can be used to reliably identify fall-related incident hip 
fractures from the person-identifying HDD when DOI is 
unavailable. 
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ARTICLE SUMMARY 



Article focus 

■ To determine the accuracy of evidence-based cri- 
teria for identifying fall-related incident hip frac- 
tures from person-identifying hospital discharge 
data in the absence of the date of injury. 

Key messages 

■ The criteria for identifying fall-related incident hip 
fractures from person-identifying hospital dis- 
charge data in the absence of date of injury were 
highly accurate when compared with the reference 
standard which contained the date of injury. 

■ The high accuracy of the criteria raises the ques- 
tion: is it really necessary to capture the date of 
injury in hip fracture hospital discharge data? 
The answer depends on a number of considera- 
tions; the majority of which relate to the quality 
and the reliability of hospital discharge data. It 
would still be ideal, however, to collect the date 
of injury for hip fracture, and other injury 
causes, in hospital discharge data as this would 
facilitate clinical and health services research. 

■ The New Zealand hospital discharge data contain a 
unique national person identifier which enables the 
identification of the same patient across multiple 
admissions. In many health systems, however, 
hospital discharge data contain no such identifier. 
Before the criteria can be applied in this situation, 
data linkage may be used to obtain person-identi- 
fying hospital discharge data. 

Strengths and limitations of this study 

■ Previous studies investigating the accuracy of arbi- 
trary, commonly used criteria for identifying inci- 
dent hip fractures from hospital discharge data did 
not employ a reference standard containing date of 
injury. Our study is therefore the first to examine 
the accuracy of evidence-based criteria in compari- 
son with the reference standard for identifying inci- 
dent hip fractures using date of injury. 

■ Our study was performed using hospital dis- 
charge data from only one health system. Future 
research could further evaluate the criteria using 
hospital discharge data from other health 
systems, which capture the date of injury. 



Vu T, Davie G, Barson D, etal. BMJ Open 201 3;3:e003222. doi:1 0.1 136/bmjopen-201 3-003222 



1 



Accuracy of evidence-based criteria for identifying an incident hip fracture 



BACKGROUND 

Hip fractures are a major injury burden in older people, 
with the majority of cases being fall-related and involving 
community dwellers. 1 2 High initial hospitalisation cost 
has been reported in many countries, including the UK, 
Belgium, the USA and Australia. 3-6 Costs of rehabilita- 
tion, home nursing, home help and institutional care 
may be substantial depending on subsequent functional 
impairment. 6 7 Patients with a hip fracture have also 
been found to have a higher risk of dying in the subse- 
quent 20 years compared with the general population. 8 9 

Accurate estimates of hip fracture incidence are 
required in the planning, delivery and prioritisation of 
service provision, and in the planning, implementation 
and evaluation of appropriate preventive interventions 
for at-risk groups. 10 Hospital discharge data (HDD) are 
an appropriate data source for estimating hip fracture 
incidence because virtually all hip fractures necessitate 
hospital admission. 11 The accurate estimation of hip 
fracture incidence in many health systems, however, is 
hampered by the lack of a unique system-wide patient 
identifier (UPI) and/or the absence of date of injury. 12 
In health systems where a UPI is lacking, HDD refers to 
specific episodes of care rather than to cases or persons. 
Data linkage may be used to identify the same patient 
across different admissions and create a person- 
identifying dataset. However, in the absence of date of 
injury, researchers' ability to distinguish multiple admis- 
sions for the same hip fracture from new admissions 
associated with a different hip fracture in the same 
person is limited. 12 

A common approach for distinguishing an admission 
due to an incident hip fracture from a readmission 
attributable to an old hip fracture is the application of a 
clearance period, commonly assumed to be 28 or 
30 days. 12-14 However, the clearance period is arbitrary 
and ignores individual fall and injury risk profiles. While 
the risk of readmission is high during the first 30 days 
following discharge, only 12-34% of all relevant readmis- 
sions occur during this period. 15-17 Thus, more than 
two-thirds of readmissions would be counted as incident 
cases resulting in a significant overestimation of true 
incidence estimates. 

In order to reliably differentiate readmissions due to 
an 'old' hip fracture from a new admission due to a sub- 
sequent hip fracture, a person-identifying data source, 
which captures the date of injury, is required. As far as 
we are aware, very few countries in the world capture 
this information in their HDD (as opposed to dedicated 
trauma or specialty registers which may capture this 
information). 18-20 In countries with no capture of this 
information in their HDD, various approaches for elim- 
inating duplicate records have been explored to increase 
the accuracy of incident hip-fracture identification. 10 12 
To the best of our knowledge, these approaches have 
not been validated against a data source which contains 
the date of injury. We modified the current approach of 
using a clearance period of 28 or 30 days by including 



additional, evidence-based criteria for incident hip frac- 
tures 21 and estimated the accuracy of our criteria using 
data containing the date of injury as the reference 
standard. 



METHODS 

We analysed New Zealand (NZ) person-identifying HDD 
which contain records of all publicly funded inpatient 
treatment of injuries in NZ. Almost 100% of all hospital 
injury treatment was publicly funded in 2006-2007 
(Chris Lewis, NZ Ministry of Health, personal communi- 
cation 2010). The data were obtained from the NZ 
Health Information Directorate, and maintained and 
enhanced by the Injury Prevention Research Unit, 
University of Otago, Dunedin, NZ. Discharges that 
occurred between 1 July 2005 and 30 June 2008 (inclu- 
sive) were used in the analysis — records were selected if 
the patient's age at admission was 65+ years, the cause of 
injury was a fall (W00-W19 in the International 
Classification of Diseases, 10th Revision, Australian 
Modification (ICD-10-AM)) and the principal diagnosis 
was a hip fracture (S72.0-S72.2 in ICD-10-AM). 22 

The selected records included the date of injury and a 
readmission indicator, and we defined this dataset as the 
reference standard. This indicator is determined by a 
computer algorithm which uses information from four 
variables in HDD, namely the unique national person 
identifier, date of admission, date of discharge and date 
of injury. The indicator has recendy been validated and 
found to have excellent accuracy between 87% and 
99%. 23 We used this indicator to exclude readmissions 
and estimate the number of fall-related incident hip 
fractures and the number of patients with multiple fall- 
related hip fractures between 1 July 2005 and 30 June 
2008, inclusive. 

We omitted the date of injury in the reference stand- 
ard dataset to form a comparison dataset on which cri- 
teria for incident hip fractures based on our review of 
relevant hip fracture literature 21 were applied (the base 
case; see table 1). Multiple criteria were developed 
based on well-documented issues of identifying repeat 
admissions for the same injury from HDD without the 
date of injury or a readmission indicator. 24 25 These cri- 
teria defined an incident hip fracture as an acute hos- 
pital admission and not an interhospital transfer. Cases 
which satisfied this definition but which only had hip 
revision procedure code(s) listed in their records were 
excluded. 26 However, cases with one or more hip oper- 
ation procedure codes in addition to one or more hip 
revision procedure codes were retained because they are 
likely to represent incident hip fractures subsequent to 
previous hip fractures. The hip operation and revision 
procedure codes used in this study are available on 
request. 

We developed additional selection criteria in order to 
distinguish between the first and subsequent fall-related 
hip fracture in the same patients due to the inaccuracy 
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Table 1 Fall-related hip fracture incident case identification by data source 



Reference standard 
dataset 



Comparison 
dataset 



Inclusion criteria 
Date of discharge from 1 July 2005 to 30 June 2008 
Principal diagnosis hip fracture (S72.0-S72.2 in ICD-1 0-AM) 
First-listed external cause indicating a fall (W00-W19 in ICD-1 0-AM) 
Age at admission 65+ years 
Exclusion criteria 
All hip fractures 

Readmission indicator positive 

Transfer from another facility 

Hip revision procedures only 

Admission type non-acute 
Additional criteria for subsequent hip fractures 

Readmission within 120 days of discharge 

All first-listed external cause codes being identical to those in the 

previous hip fracture 



/ 
/ 
/ 
/ 



/ 
/ 
/ 
/ 



/ 
/ 
/ 

/ 



ICD-1 0-AM, International Classification of Diseases, 10th Revision, Australian Modification. 



of fracture type classification 28 and the lack of 
ICD-1 0-AM codes on laterality of fractures (Patricia 
Saad, National Centre for Classification in Health 
(Australia), personal communication 2010). We assumed 
that the minimum clearance period between the dis- 
charge date of the index episode of care for the first 
incident hip fracture and the admission date of the 
index episode of care for the second incident hip frac- 
ture would be 120 days, 29 3 and that all first-listed exter- 
nal cause of injury codes (mechanism of injury, place of 
occurrence and activity being undertaken when injured) 
would differ between 'old' and 'new' hip fractures 
belonging to the same patient. Given that external cause 
of injury is detailed in ICD-10-AM, it would appear that 
the probability of two hip fractures in the same patient 
having three identical first-listed cause codes would be 
very small. Nevertheless, we tested this assumption in a 
sensitivity analysis and also investigated the completeness 
or otherwise of external cause information in the refer- 
ence standard dataset. 

We applied the above criteria sequentially according 
to the order listed in table 1. We calculated the sensitiv- 
ity, specificity, positive and negative predictive values of 
the criteria compared with the reference standard using 
standard definitions. 31 We conducted a one-way sensitiv- 
ity analysis of the base case by modifying the criteria one 
at a time. Eight scenarios were considered: 

1. SI included interhospital transfers 

2. S2 included cases with only hip revision procedures 

3. S3 included non-acute admissions 

4. S4 modified the clearance period to 30 days 

5. S5 modified the clearance period to 60 days 

6. S6 modified the clearance period to 90 days 

7. S7 omitted the criterion regarding external cause 
codes 

8. S8 omitted the criterion regarding the clearance 
period 



We conducted all analyses in Stata V.ll. We evalu- 
ated equality of proportions using two-sample % 2 teste of 
proportions or Fisher's exact test, as appropriate. For 
skewed continuous variables, we compared medians 
using non-parametric K-sample teste on the equality of 
medians. 



RESULTS 

Fall-related hip fracture in older people in NZ according to 
the reference standard dataset 

The reference standard dataset contained 9637 episodes 
associated with 8761 patients. A total of 787 episodes of 
care were classified as readmissions by the readmission 
indicator and hence excluded, resulting in an estimate 
of 8850 fall-related incident hip fractures in 8524 
patients between 1 July 2005 and 30 June 2008. Among 
these patients, 8200 (96.2%) had one fall-related hip 
fracture, 322 (3.8%) had two fall-related hip fractures 
and 2 (<0.1%) had three fall-related hip fractures. Fewer 
than 1% of the fall-related incident hip fractures were 
admitted as non-acute cases and a similar percentage of 
cases were coded as interhospital transfers. Overall, 6% 
of patients had at least one readmission (range 1-3) . 

The median time from hospital discharge after treat- 
ment for one fall-related hip fracture to admission for 
treatment of another fall-related hip fracture was 
203 days (interquartile range 71-441). 

A total of 1054 fall-related incident hip fractures 
(11.9% of total cases) belonging to 1039 patients did 
not receive any surgical interventions. Fourteen cases 
(in 14 patients) had both hip revision and hip operation 
procedures. Of the cases without surgical intervention, 
3% had at least one hip revision procedure code sug- 
gesting the possibility of coding error. 

In patients with multiple fall-related hip fractures, 212 
(65%) sustained a subsequent fall-related hip fracture at 
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the same place as the previous fall-related hip fracture. 
Few subsequent fall-related hip fractures (28 or 8.6% of 
all refracture cases) had all first-listed external cause 
codes identical to those in the preceding fall-related hip 
fracture. The presence of an unspecified element in all 
first-listed external cause codes was rare (1.8% of all 
cases) . 

Accuracy of criteria for identifying fall-related incident hip 
fractures in the absence of date of injury 

Our criteria for identifying fall-related incident hip frac- 
tures in the absence of date of injury compared favour- 
ably with the reference standard. The sensitivity and 
specificity of the base case criteria were 96.7% and 
99.3%, respectively (table 2). These criteria failed to 
detect 3.3% of cases (false negatives). Of the false nega- 
tives, 244 (84.4%) were first fall-related hip fractures and 
45 (15.6%) were second fall-related hip fractures. More 
than one criterion was responsible for the exclusion of 
22.1% of the former and 100% of the latter (table 3). 

The one-way sensitivity analysis (table 2) suggests that 
all accuracy measures estimated for the base case are 
robust. Accepting interhospital transfers (SI), including 
cases with hip revision (s) alone (S2) and accepting non- 
acute admissions (S3) reduced false negative cases from 
3.3% to 2.7%, 2.9% and 2.7%, respectively. However, the 
modification in SI, S2 and S3 had little effect on the 
accuracy measures of each of these scenarios and only 
minor accuracy differences separated them. 

The inclusion of cases irrespective of whether or not 
they had the same first-listed external cause codes as the 
preceding fall-related hip fracture (S7) resulted in accur- 
acy estimates being identical or near identical to those 
in the base case, suggesting that this criterion was not a 
key driver of accuracy. The modification of the clearance 
period in S4-S6 revealed that a clearance period greater 
than 30 days (S4) , in conjunction with all other criteria, 
was optimal based on a combination of measures: the 
percentage of false negatives (2.3%), the percentage of 



patients with multiple fall-related hip fractures (3.2%) 
and accuracy estimates (between 98% and 99%). 

DISCUSSION 

The criteria for identifying fall-related incident hip frac- 
tures from person-identifying HDD in the absence of 
date of injury were highly accurate when compared with 
the reference standard. Our findings clearly demon- 
strate that when the date of injury is not available in 
person-identifying HDD, which is the situation in many 
countries, evidence-based criteria can be used to reliably 
identify fall-related incident hip fractures. This is the 
first published study to investigate the accuracy of 
evidence-based criteria in comparison to a gold standard 
of identifying fall-related incident hip fractures using the 
date of injury. Two previous studies compared commonly 
used criteria for identifying incident hip fractures from 
HDD but neither included this gold standard. 10 12 

The high accuracy of the criteria for identifying fall- 
related incident hip fractures raises the question: is it 
really necessary to capture the date of injury in hip frac- 
ture HDD? The answer depends on a number of consid- 
erations. First, NZ patients with hip fracture may or may 
not be representative of all patients with this condition 
in terms of the clinical and admission characteristics 
incorporated into these criteria. These characteristics, in 
turn, may be related to patients' age, and hip fracture 
admission policies and treatment practices in a specific 
health system. Second, the performance of our criteria 
depends on the reliability of coded HDD. The misclassi- 
fication error in surgical procedure codes in NZ HDD 
would appear to be small, since the surgical intervention 
rate is within the range of rates reported in the litera- 
ture 15 17 33 and only 3% of incident hip fractures were 
coded with one or more hip revision procedure and no 
hip operation procedure. In contrast, Dalai and Roy 34 
reported significant coding errors in hip fracture 
surgery between 24% and 32.8% in one UK hospital. 
The completeness and specificity of first-listed external 



able 2 Accuracy of evidence-based criteria for identifying fall-related incident hip fractures without the date of injury 



Accuracy* 


Base case 


S1 accepted 
interhospital 
transfers 


S2 accepted hip 
revision(s) alone 


S3 accepted 
non-acute admissions 


S4 accepted 
clearance period 
>30 days 


Sensitivity 
Specificity 
PPV 
NPV 


96.7 (96.3 to 97.1) 
99.3 (99.2 to 99.4) 
98.3 (98.0 to 98.6) 
98.7 (98.6 to 98.9) 


97.3 (97.0 to 97.7) 
99.2 (99.1 to 99.3) 
97.9 (97.6 to 98.2) 
99.0 (98.8 to 99.1) 


97.1 (96.7 to 97.4) 
99.3 (99.2 to 99.4) 
98.3 (98.0 to 98.6) 
98.9 (98.7 to 99.0) 


97.3 (97.0 to 97.6) 
99.3 (99.1 to 99.4) 
98.1 (97.8 to 98.3) 
99.0 (98.8 to 99.1) 


97.7 (97.4 to 98.0) 
99.4 (99.2 to 99.5) 
98.3 (98.0 to 98.6) 
99.1 (99.0 to 99.2) 


Accuracy* 


Base case 


S5 clearance 
period 60 days 


S6 clearance 
period 90 days 


S7 no criterion on 
external cause codes 


S8 no criterion on 
clearance period 


Sensitivity 
Specificity 
PPV 
NPV 


96.7 (96.3 to 97.1) 
99.3 (99.2 to 99.4) 
98.3 (98.0 to 98.6) 
98.7 (98.6 to 98.9) 


97.3 (97.0 to 97.7 
99.3 (99.2 to 99.4) 
98.3 (98.0 to 98.6) 
99.0 (98.8 to 99.1) 


97.0 (96.6 to 97.4) 
99.3 (99.2 to 99.4) 
98.3 (98.0 to 98.6) 
98.8 (98.7 to 99.0) 


96.9 (96.2 to 97.2) 
99.4 (99.2 to 99.5) 
98.3 (98.0 to 98.6) 
98.8 (98.6 to 98.9) 


98.0 (97.7 to 98.3) 

98.7 (98.6 to 98.9) 

96.8 (96.4 to 97.2) 
99.2 (99.1 to 99.3) 



'Values are percentages (95% CIs). 

NPV, negative predictive value; PPV, positive predictive value. 
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Table 3 Reason(s) for the exclusion of false negatives 


Reason, 
n (%) 


Excluded first hip fractures (n=244) 






Transfer from another hospital 


90 


(36.9) 


Non-acute admission 


73 


(29.9) 


No hip operation 


27 


(11.1) 


Transfer from another hospital and 


54 


(22.1) 


non-acute admission 






Excluded second hip fractures (n=54) 






Transfer from another hospital and 


54 


(100) 


non-acute admission, and time to refracture 






<120 days and first-listed external cause 






codes identical to a previously admitted hip 






fracture 









cause of injury codes in HDD may also require consider- 
ation. The level of missing, first-listed external causes of 
injury in the NZ hip fracture HDD used was negligible 
(<0.1%), whereas the level of missing external causes of 
injury in other HDD is reportedly high. 35 The percent- 
age of incident cases in the NZ hip fracture HDD with 
all first-listed external cause codes denoting unspecified 
elements was small (<2.0%). If this percentage is signifi- 
cantly elevated, the external cause criterion may be ren- 
dered less effective in distinguishing multiple hip 
fractures in the same patients. 

Our analysis of the reasons behind the exclusion of 
false negative cases by the base case suggests that some 
of these cases may be atypical presentations of fall- 
related incident hip fractures, such as elective admis- 
sions rather than being admitted as an emergency case 
or hip operation procedure (s) not being performed due 
to patients' frailty or a terminal illness. Additionally, 
some patients might have been transferred from 
another hospital on the same day without having the 
first episode of care at the originating hospital being 
recorded. It should be noted that the study population 
was comprised of both community-dwelling older 
people and nursing home residents. The latter group is 
frail and/ or disabled, and is commonly affected by a 
range of comorbidity 36 ; hence, they may have received 
palliative care and/or other type of non-acute care 
rather than surgical intervention (s) for their hip frac- 
tures. However, we were unable to disaggregate our 
results by community-dwelling status to confirm this. 

The one-way sensitivity analysis suggests that the base 
case may be improved by reducing the clearance period 
from 120 to 30 days, a period commonly used to exclude 
readmissions but which until now had not been vali- 
dated by population-level dataset. This finding appears 
to contradict previous research which suggests that only 
12-34% of all relevant readmissions occur during this 
period. 15-17 However, it is important to note that we did 
not solely use the clearance period to identify incident 
hip fractures; it was only one of a set of criteria used to 
differentiate readmissions from incident hip fractures. 



Finally, the sensitivity analysis found that the inclusion of 
interhospital transfers (SI) did not alter the accuracy of 
the base case, suggesting that transfers may be counted 
as incident cases. This finding should be interpreted 
with caution because the SI scenario contains other cri- 
teria in addition to that for transfers. Furthermore, the 
transfers identified by the reference standard as incident 
cases have been verified by the date of injury. It is well 
established that the exclusion of transfers would minim- 
ise the overestimation of the incidence of hospitalised 

• • 12 15 

injury. 

Our study was performed using only one national 
injury hospital discharge dataset. The NZ data contain a 
unique national person identifier which enables the 
identification of the same patient across multiple admis- 
sions. In many health systems, however, HDD contain no 
such identifier. 12 37 Before our criteria can be applied in 
this situation, data linkage may be used to obtain 
person-identifying HDD. Future research could further 
evaluate the criteria using other person-identifying HDD 
datasets that contain the date of injury. Hospital admis- 
sion policies and treatment practices for hip fractures 
might differ between countries and this may translate 
into different admission patterns compared with those 
in NZ. Moreover, almost 100% of all hospital injury treat- 
ment is publicly funded in NZ, whereas in many other 
countries the public as well as private hospital sectors 
are involved in the provision of hospital treatment for 
injuries. Accordingly, the reconciliation of episodes of 
care belonging to the same patients may be easier in the 
former compared with the latter. Ultimately, admission 
patterns reflect underlying injury incidence rates and 
injury severity in older people, and variations in hospital 
utilisation for those injuries. 

Our study demonstrates a valid approach for identify- 
ing fall-related incident hip fractures in older people 
from person-identifying HDD in the absence of date of 
injury. It would still be ideal, however, to collect the date 
of injury in HDD as this would enable clinical and 
health services research questions to be addressed for all 
injury causes. 



CONCLUSIONS 

Few countries capture the date of injury in their HDD. 
It has previously been suggested that the lack of this 
date has hampered the accurate identification of inci- 
dent hip fractures from HDD. Our study has shown 
that in the absence of date of injury, evidence-based 
criteria may be used to accurately identify fall-related 
incident hip fractures in older people from person- 
identifying HDD. However, the collection of date of 
injury for inclusion in hip fracture HDD might still be 
necessary unless the quality and reliability of coded 
HDD can be established. Furthermore, the date of 
injury would facilitate clinical and health services 
research for all injury causes. 
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